Information processing apparatus, information processing method, and program

ABSTRACT

Provided is an information processing apparatus including an image supply unit that supplies a plurality of input images showing corresponding objects to an image processing unit and obtains a plurality of object images as an image processed result from the image processing unit, and a display control unit that synchronously displays the plurality of object images that have been obtained. The object images are regions including the corresponding objects extracted from the plurality of input images, and orientations, positions, and sizes of the corresponding objects of the plurality of object images are unified.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/424,222, filed on Feb. 26, 2015, which is a national phaseentry under § 371 of International Application No. PCT/JP2013/072594,filed on Aug. 23, 2013, published as WO 2014/038408 on Mar. 13, 2014,which claims priority from Japanese Patent Application No. 2012-196008,filed in the Japanese Patent Office on Sep. 6, 2012, the disclosures ofwhich are hereby incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a program, in particular, to aninformation processing apparatus, an information processing method, anda program that are suitable for simultaneously displaying a plurality ofpathological images so as to comparatively perform pathologic diagnosis.

BACKGROUND ART

At ordinary health care sites, as methods that diagnose pathologicaltissues such as tumors (for example, diagnose whether a pathologicaltissue is a malignant tumor such as a cancer), a prepared specimen ismade in such a manner that part of a pathological tissue is harvestedfrom the patient, the harvested pathological tissue is sliced, thesliced tissue (hereinafter referred to as a biopsy) is placed on a slideglass, and then the biopsy is stained. The prepared specimen is observedand diagnosed under a microscope or the like.

The following relationships of staining methods (such as reagents),staining targets, and staining colors are known.

Staining Method Staining Target Color Hematoxylin cell nucleus bluishpurple Eosin cell nucleus, light red connective tissue PAS stainingmucus purple KB staining nerve fiber blue Keratin 903 basal cell brown

Besides above staining methods, Hematoxylin & Eosin (H&E),Immunohistochemistry (IHC), Fluorescence In Situ Hybridization (FISH),and so forth are known.

H&E can be used to stain basophilic cell nucleus, bony tissue, part ofcartilaginous tissue, serum component, and so forth in bluish purple;and acidophilic cytoplasm, connective tissue of soft tissue,erythrocyte, fibrin, endocrine granule, and so forth in red pink.

IHC can be used to visualize an antigen-antibody reaction. FISH can beused to map a gene and detect chromosome aberrations.

Thus, since these staining methods deal with different staining targets,when different staining methods are applied to the same part of thetissue and the resultant part is observed, diagnosis can be moreaccurately perform than one staining method is applied.

However, if a sliced tissue on one prepared specimen is stained by aplurality of different staining methods, since stained colors are notproperly developed, the diagnosis is likely to become difficult. Inaddition, some staining methods may not be used together. Thus, it ispreferred to prevent a plurality of staining colors from co-existing onone prepared specimen.

FIG. 1 shows an example prepared specimen that has been made. A preparedspecimen P1 shown in FIG. 1A and a prepared specimen P2 shown in FIG. 1Bare composed of biopsies adjacently cut from a harvested pathologicaltissue. For example, it is assumed that one of the prepared specimens P1and P2 is stained by H&E, the other is stained by IHC.

A biopsy b11 at the left end on the prepared specimen P1 shown in FIG.1A and a biopsy b12 at the left end on the prepared specimen P2 shown inFIG. 1B are adjacently cut from the pathological tissue that ispunctured and harvested. Hereinafter, the relationship between thebiopsy b11 and the biopsy b21 is referred to as corresponding biopsies.Likewise, the relationship between a biopsy b21 at the center of FIG. 1Aand a biopsy b22 at the center of FIG. 1B and the relationship between abiopsy b31 at the right end of FIG. 1A and a biopsy b32 at the right endof FIG. 1B are also respectively referred to as corresponding biopsies.

Rectangle frames on the prepared specimens P1 and P2 represent regionson the same coordinates on the prepared specimens P1 and P2. When theprepared specimens P1 and P2 are compared, it is obvious thatcorresponding biopsies on the two prepared specimens P1 and P2 are notalways located on the same coordinates. In addition, when a biopsy iscut, the shape of the biopsy may be deformed depending on the cuttingforce and so forth applied to the biopsy.

As the simplest method that compares the two prepared specimens P1 andP2, quickly moving the observing positions of the two prepared specimensP1 and P2 simultaneously placed under a microscope, a diagnostician suchas a pathologist looks for corresponding portions and performsdiagnosis. In this case, however, if the diagnostician moves theprepared specimens excessively or insufficiently, it is difficult forhim or her to accurately and efficiently observe the correspondingportions.

Thus, as a method that does not cause the diagnostician to move theprepared specimens, a virtual microscope system has been proposed (forexample, see Patent Literature 1 below).

The virtual microscope system divides a diagnosing biopsy on a preparedspecimen into small regions, photographs the divided small regionsthrough an objective lens having a high resolution, and combines theplurality of small regions so as to reconstruct the image of thediagnosing biopsy as digital image data.

When two prepared specimens are reconstructed as digital image data bythe virtual microscope system, the two prepared specimens can besimultaneously displayed on a screen or the like of a personal computer.

CITATION LIST Patent Literature

Patent Literature 1: JP H09-281405A

SUMMARY OF INVENTION Technical Problem

However, even if such a virtual microscope system is used to accuratelyand simultaneously display corresponding portions on two preparedspecimens, the diagnostician still needs to perform operations of thesystem including an enlargement operation, a reduction operation, and arotation operation for images.

The present disclosure is made from the foregoing point of view suchthat corresponding portions of a plurality of images are accurately andsimultaneously displayed.

Solution to Problem

An information processing apparatus according to an aspect of thepresent disclosure includes an image supply unit that supplies aplurality of input images showing corresponding objects to an imageprocessing unit and obtains a plurality of object images as an imageprocessed result from the image processing unit, and a display controlunit that synchronously displays the plurality of object images thathave been obtained. The object images are regions including thecorresponding objects extracted from the plurality of input images, andorientations, positions, and sizes of the corresponding objects of theplurality of object images are unified.

The display control unit may execute at least one of a guide scrolldisplay, an automatic vertical/horizontal division selection display, aninstantaneous switch display, a cut and placement display, aleaf-through display, and a multiple staining color combining display.

The plurality of images may be medical images.

The objects may be biopsies cut from a tissue. The plurality of medicalimages may be pathological images that are scanned from preparedspecimens of which the biopsies adjacently cut from a same tissue andplaced on slide glasses and stained by different staining methods.

The image processing unit may be a server located on Internet.

The information processing apparatus according to an aspect of thepresent disclosure may further include the image processing unit.

An information processing method according to an aspect of the presentdisclosure performed by an information processing apparatus includessupplying a plurality of input images showing corresponding objects toan image processing unit, obtaining a plurality of object images as animage processed result from the image processing unit, and synchronouslydisplaying the plurality of object images that have been obtained. Theobject images are regions including the corresponding objects extractedfrom the plurality of input images, and orientations, positions, andsizes of the corresponding objects of the plurality of object images areunified.

A program according to an aspect of the present disclosure for causing acomputer to function as an image supply unit that supplies a pluralityof input images showing corresponding objects to an image processingunit and obtains a plurality of object images as an image processedresult from the image processing unit, and a display control unit thatsynchronously displays the plurality of object images that have beenobtained. The object images are regions including the correspondingobjects extracted from the plurality of input images, and orientations,positions, and sizes of the corresponding objects of the plurality ofobject images are unified.

According to an aspect of the present disclosure, a plurality of inputimages that show corresponding objects are supplied to an imageprocessing unit, a plurality of object images are obtained as an imageprocessed result of the image processing unit, and the plurality of theobtained object images are synchronously displayed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing example prepared specimens.

FIG. 2 is a block diagram showing an example configuration of apathological image display control device according to an embodiment ofthe present disclosure.

FIG. 3 is a schematic diagram showing example pathological images.

FIG. 4 is a block diagram showing an example configuration of a biopsyregion image generation server.

FIG. 5 is a schematic diagram describing a procedure that generates adetection dictionary.

FIG. 6 is a schematic diagram describing the detection of a cellulartissue region, the grouping of tissue regions, and the cutting of thetissue regions.

FIG. 7 is an enlarged view showing biopsy region images that have beencut.

FIG. 8 is a schematic diagram describing the corrections of thepositions and sizes of biopsy region images.

FIG. 9 is an enlarged view of corrected biopsy region images.

FIG. 10 is a flow chart describing a pathological image synchronousdisplay process.

FIG. 11 is a schematic diagram showing an example screen shot of anautomatic vertical/horizontal division selection display.

FIG. 12 is a schematic diagram showing an example screen shot of aninstantaneous switch display.

FIG. 13 is a schematic diagram showing an example screen shot of a cutand placement display.

FIG. 14 is a schematic diagram showing an example screen shot of aleaf-through display.

FIG. 15 is a schematic diagram showing an example screen shot of amultiple staining color combining display.

FIG. 16 is a schematic diagram showing an example screen shot of arelevant comment display.

FIG. 17 is a block diagram showing an example configuration of acomputer.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the drawings, elements that have substantiallythe same function and structure are denoted with the same referencesigns, and repeated explanation is omitted.

[Example Configuration of Pathological Image Display Control Device]

A pathological image display control device corresponds to aninformation processing apparatus according to an embodiment of thepresent disclosure. The pathological image display control deviceaccurately and simultaneously displays corresponding portions of aplurality of pathological images that show biopsies adjacently cut froma harvested pathological tissue.

In the following example, the case that two pathological images aresimultaneously displayed. However, the present disclosure can be appliedto the case that three or more pathological images are simultaneouslydisplayed.

Here, the pathological image represents digital image data that is usedto perform diagnosis and that is read by a dedicated scanner from aprepared specimen made of a biopsy or sample harvested from a tissue offor example a human body.

Besides pathological images, the present disclosure can be also appliedto medical images of human bodies captured through a CT, an MRI, an Xray, or the like and any non-medical images that are simultaneouslydisplayed.

FIG. 2 shows an example configuration of the pathological image displaycontrol device corresponding to the information processing apparatusaccording to the present disclosure.

The pathological image display control device 10 is composed of anoperation input unit 11, a pathological image input unit 12, a biopsyregion image obtainment unit 13, and a display control unit 14.

The operation input unit 11 accepts a selection operation forpathological images, various display operations, and so forth from auser (diagnostician) and outputs corresponding operation signals to thepathological image input unit 12 or the display control unit 14.

The pathological image input unit 12 inputs two pathological images PP1and PP2 that show corresponding biopsies used for comparative diagnosisof prepared pathological images to the biopsy region image obtainmentunit 13 corresponding to an operation signal based on the user'sselection operation.

FIG. 3 shows an example of the two pathological images PP1 and PP2 thatare input from the pathological image input unit 12 to the biopsy regionimage obtainment unit 13. These pathological images PP1 and PP2 arescanned from prepared specimens P1 and P2 shown in FIG. 1 by a scanneror the like.

The biopsy region image obtainment unit 13 sends the two pathologicalimages to a biopsy region image generation server 20. In addition, thebiopsy region image obtainment unit 13 obtains biopsy region imagesgenerated based on the two pathological images PP1 and PP2 from thebiopsy region image generation server 20 and supplies the biopsy regionimages to the display control unit 14.

Although the biopsy region image generation server 20 is located forexample on the Internet, all or part of the biopsy region imagegeneration server 20 may be built in the pathological image displaycontrol device 10.

The biopsy region images represent images generated in such a mannerthat cellular tissue regions are detected from a pathological image (theentire image on the prepared specimen), the detected cellular tissueregions are grouped and extracted corresponding to the individualbiopsies, and the orientations and sizes of the extracted cellulartissue regions are corrected and unified.

Corresponding to the user's operation, the display control unit 14simultaneously displays the two biopsy region images supplied from thebiopsy region image obtainment unit 13 on a display 30 locateddownstream of the display control unit 14. Hereinafter, the operationthat causes biopsy region images to be simultaneously displayed is alsoreferred to as the synchronous display. Various synchronous displaymethods that the display control unit 14 performs will be describedlater with reference to FIGS. 11 to 16.

FIG. 4 shows an example configuration of the biopsy region imagegeneration server 20.

The biopsy region image generation server 20 is composed of a cellulartissue region detection unit 21, a detection dictionary 22, a biopsyregion grouping unit 23, a biopsy region cut unit 24, and a positioncorrection unit 25.

Consulting the detection dictionary 22, the cellular tissue regiondetection unit 21 detects cellular tissue regions from the entireregions of the pathological images PP1 and PP2 shown in FIG. 3 (namely,the regions of the biopsies b11, b21, and b31 shown in FIG. 3A and theregions of the biopsies b12, b22, and b32 shown in FIG. 3B).

The detection dictionary 22 is pre-generated through statisticallearning using learning cellular tissue region images. The generation ofthe detection dictionary 22 will be described with reference to FIG. 5.

A patch image whose center is a cellular tissue region is cut from alearning cellular tissue region image and a patch image whose center isa background portion (non-cellar tissue region) are prepared as alearning data set. The image feature amounts of the learning data setare extracted. The image whose center is the cellular tissue region andthe image whose center is the background portion are statisticallylearned as positive data and negative data, respectively.

Although the image feature amount of a patch image can be extracted byany method, the PixDif luminance differential feature amount thatcalculates the luminance difference of any two points on a patch imagemay be used as disclosed in for example JP 2005-284348A. Likewise, anystatistic learning method for example Boosting may be also applied.

When the detection dictionary 22 is used, a final hypothesis F for aninput patch image x can be given by Formula (1) that follows where f isa learned weak hypothesis, α is a weight, and the number of f's is T.

$\begin{matrix}{\left\lbrack {{Math}\mspace{14mu} 1} \right\rbrack\mspace{661mu}} & \; \\{{F(x)} = {\overset{T}{\sum\limits_{t}}{\alpha_{t}{f_{t}(x)}}}} & (1)\end{matrix}$

Thus, the cellular tissue region detection unit 21 cuts the input patchimages whose centers are all the pixels of the pathological images PP1and PP2, calculates the final hypothesis F for the input patch images,and performs a threshold process for the values of F so as to determinewhether the center pixels of the input patch images are a positiveregion (cellular tissue region) or a negative region (non-cellulartissue region).

The results of the cellular tissue regions that the cellular tissueregion detection unit 21 detects are as shown in FIG. 6B.

Returning to FIG. 4, the biopsy region grouping unit 23 groups thecellular tissue regions detected by cellular tissue region detectionunit 21 corresponding to the individual biopsies.

Specifically, pixels determined as a cellular tissue region are groupedcorresponding to individual biopsy numbers that represent biopsies inthe pathological images. The number of biopsies in the pathologicalimages is known when prepared specimens are made. Thus, this groupingbecomes a clustering problem in which the number of clusters is known.Next, a clustering method using spectrum clustering will be described.

It is assumed that the number of pixels in the cellular tissue region isdenoted by n; the number of clusters of the target grouping (the numberof biopsies) is denoted by C; and the Euclidean distance as coordinatevalues of a pixel i and a pixel j is denoted by d_(ij).

In addition, an affinity matrix A_(ij) is defined as Formula (2) thatfollows.

$\begin{matrix}{\left\lbrack {{Math}\mspace{14mu} 2} \right\rbrack\mspace{661mu}} & \; \\{{A_{ij} = {{\exp\left( \frac{- d_{ij}^{2}}{\sigma^{2}} \right)}\left( {1 \neq j} \right)}}{A_{ii} = 0}} & (2)\end{matrix}$

where σ is a scale parameter that is an appropriate value (for example,0.1).

Next, a diagonal matrix D is defined as Formula (3) that follows so asto obtain a matrix L.

$\begin{matrix}{\left\lbrack {{Math}\mspace{14mu} 3} \right\rbrack\mspace{661mu}} & \; \\{{D_{ii} = {\sum\limits_{j = 1}^{n}A_{ij}}}{L = {D^{- \frac{1}{2}}{AD}^{- \frac{1}{2}}}}} & (3)\end{matrix}$

Next, C eigenvectors x₁, x₂, . . . , x_(c) are obtained in thedescending order of eigenvalues of the matrix L to generate a matrixX=[x₁, x₂, . . . , x_(c)]. Thereafter, a matrix Y of which X isnormalized in each row is obtained by Formula (4) that follows.

$\begin{matrix}{\left\lbrack {{Math}\mspace{14mu} 4}\; \right\rbrack\mspace{655mu}} & \; \\{Y_{ij} = \frac{X_{ij}}{\left( {\sum\limits_{j}X_{ij}^{2}} \right)^{\frac{1}{2}}}} & (4)\end{matrix}$

When each row of the matrix Y is clustered to C element vectors byK-means, a cluster having a row number i of the matrix Y corresponds toa cluster of a pixel i.

Besides spectral clustering, the biopsy region grouping unit 23 mayperform grouping using any clustering technique that directly appliesK-means to input data. In this case, an appropriate clustering techniqueis preferably used corresponding to the characteristic of input data.

The grouped result of the biopsy region grouping unit 23 is for example,as shown in FIG. 6C, three groups that are the same as the number ofbiopsies, which are also the same as the number of cellular tissueregions shown in FIG. 6B.

Returning to FIG. 4 again, the biopsy region cut unit 24 corrects therotation of the pathological image of each grouped biopsy region andcuts the corrected image.

When the rotation of the pathological image of each grouped biopsyregion is corrected, a slope θ of the principal axis of inertia is givenby Formula (5) that follows where a p-order moment on the x axis aroundthe center of gravity and a q-order moment on the y axis around thecenter of gravity are denoted by u_(pq).

$\begin{matrix}{\left\lbrack {{Math}\mspace{14mu} 5} \right\rbrack\mspace{661mu}} & \; \\{\theta = {\frac{1}{2}{\tan^{- 1}\left( \frac{2\; u_{11}}{u_{20} - u_{02}} \right)}}} & (5)\end{matrix}$

The rotation of the original pathological image is corrected by theslope θ of the principal axis of inertia. Thereafter, a white regionhaving a predetermined width (for example, several hundred pixels)around the biopsy region is cut from the pathological image whoserotation has been corrected. As a result, a biopsy region image as shownin FIG. 6D is generated.

In FIG. 6D and the later drawings, only the biopsies b11 and b12 at theleft end of the three tissues on the pathological images PP1 and PP2 areshown. However, the same process is performed for the biopsies b21 andb22 at the center of the three tissues and the biopsies b31 and b32 atthe right end of the three tissues.

FIG. 7 is an enlarged view of biopsy region images cut by the biopsyregion cut unit 24. FIG. 7A is a biopsy region image cut from the biopsyb11 at the left end of the pathological image PP1. FIG. 7B is a biopsyregion image cut from the biopsy b12 at the left end of the pathologicalimage PP2.

As is clear from FIG. 7A and FIG. 7B, the biopsy b11 and biopsy b12slightly deviate from each other in the vertical direction. In addition,the sizes of the biopsy b11 and biopsy b12 vary (expanded or shrunk).These vertical deviations and size variations are likely to occurdepending on the pressure applied when the biopsies b11 and b12 are cutfrom tissues and placed on slide glasses. If the vertical deviationsand/or size variations occur in the biopsy b11 and biopsy b12, it isdifficult to accurately and simultaneously observe and diagnose thecorresponding portions of the biopsies b11 and b12.

Thus, the position correction unit 25 located immediately downstream ofthe biopsy region cut unit 24 corrects and unifies the positions andsizes of the cut biopsy region images. A specific procedure will bedescribed with reference to FIG. 8. In FIG. 8, the corrections of onlyvertical deviations and size variations of the biopsy region images aredescribed. Likewise, the corrections of horizontal deviations and sizevariations are performed.

First, binary images that distinguish a cellular tissue region and abackground (non-cellular tissue region) shown in FIG. 8B are obtainedfrom the biopsy region images shown in FIG. 8A. Thereafter, the numberof pixels in the cellular tissue region at each vertical coordinateposition of the binary images is counted. Thereafter, as shown in FIG.8C, histograms having a vertical axis that represents the verticalcoordinate of each of binary images and a horizontal axis thatrepresents the number of pixels are generated.

As shown in FIG. 8D, the two obtained histograms are superposed on eachother such that while one histogram is fixed, the vertical position ofthe other histogram is repeatedly adjusted to enlarge or reduce thevertical size until the two histograms are sufficiently superposed oneach other (specifically, a predetermined evaluation function value (forexample, an inner product value) becomes a predetermined value orgreater).

The adjustment values of the final vertical positions and enlarged orreduced correction values of the vertical sizes are applied to theoriginal biopsy region images. Thus, as shown in FIG. 9, the two biopsyregion images that deviate from each other are corrected to two biopsyregion images that do not deviate from each other. FIG. 9 shows that thetwo biopsy region images that had deviated from each other shown in FIG.7 have been corrected.

The corrected biopsy region images are returned from the biopsy regionimage generation server 20 to the biopsy region image obtainment unit13.

[Operational Description]

Next, the pathological image synchronous display process of thepathological image display control device 10 will be described withreference to FIG. 10. FIG. 10 is a flow chart describing thepathological image synchronous display process.

At step S1, the pathological image input unit 12 inputs two pathologicalimages PP1 and PP2 used for comparative diagnosis to the biopsy regionimage obtainment section 13 corresponding to an operation signal basedon a user's selective operation. The biopsy region image obtainmentsection 13 sends the two pathological images to the biopsy region imagegeneration server 20.

At step S2, consulting the detection dictionary 22, the cellular tissueregion detection unit 21 of the biopsy region image generation server 20detects cellular tissue regions from the pathological images PP1 andPP2.

At step S3, the biopsy region grouping unit 23 groups the cellulartissue regions detected by the cellular tissue region detection unit 21corresponding to the individual biopsies. The biopsy region cut unit 24corrects the rotation of each of the grouped biopsy regions based on theslope θ of the principal axis of inertia and cuts a white region havinga predetermined width (for example, several hundred pixels) around thebiopsy region from the pathological image whose rotation has beencorrected so as to generate a biopsy region image.

At step S4, the position correction unit 25 corrects the positions andsizes of the biopsy region images. The position correction unit 25returns the corrected biopsy region images from the biopsy region imagegeneration server 20 to the biopsy region image obtainment unit 13.

At step S5, the display control unit 14 causes the two biopsy regionimages supplied from the biopsy region image obtainment unit 13 to besynchronously displayed on the display 30 located downstream of thedisplay control unit 14 corresponding to a user's operation. Now, thedescription of the pathological image synchronous display process iscompleted.

[Specific Examples Synchronous Displays]

Next, examples synchronous displays of the display control unit 14 willbe described.

“Guide Scroll Display”

When a biopsy region image is synchronously displayed while the biopsyregion image is being enlarged and vertically or horizontally scrolled,the center of the display screen is moved along the shape of the biopsyregion image. For example, if the biopsy region image is formed in the“<” shape as shown in FIG. 9, when the biopsy region image is scrolleddownward, the center of the screen is moved to the left and then to theright along the shape of the biopsy region image. As a result, while theuser is scrolling only a portion necessary for diagnosis, he or she canalways observe the portion.

“Automatic Vertical/Horizontal Division selection Display”

When the shape of the biopsy region in an biopsy region image isportrait, a screen 50 is divided in the vertical direction and the twodivided biopsy region images are horizontally and simultaneouslydisplayed as shown in FIG. 11A. When the shape of the biopsy region in abiopsy region image is landscape, a screen 60 is horizontally divided inthe horizontal direction and the two divided biopsy region images arevertically and simultaneously displayed as shown in FIG. 11B. As aresult, since the biopsy region displayed on the screen is widened, theuser can easily observe and diagnose the biopsy region.

Whether the biopsy region is portrait or landscape depends on the angle(slope) of the principal axis of inertia of the biopsy region. Assumingthat the upward vertical direction is 0 degree and the counterclockwisedirection is the positive angle direction, when the angle of theprincipal axis of inertia is from −45 degrees to +45 degrees or from 130degrees to 225 degrees, the biopsy region is displayed as a portraitregion; otherwise, the biopsy region is displayed as a landscape region.

“Instantaneous Switch Display”

A biopsy region image 70 shown in FIG. 12A and a biopsy region image 71shown in FIG. 12B may be instantaneously switched between the biopsyregion image 70 and the biopsy region image 71 on the displaycorresponding to a user's predetermined operation. When this displaymethod is used, since the user can see these biopsy region images withless movement of his or her viewpoint than the user simultaneously seestwo images simultaneously displayed, he or she can easily compare thecorresponding portions of the two biopsy region images.

“Cut and Placement Display”

When the user specifies a selection region 81 having any size on abiopsy region image 80 that shows the tissue b11 as shown in FIG. 13A, acut display region 83 is displayed adjacent to the selection region 81as shown in FIG. 13B. The cut display region 83 shows a portion of thetissue b12 corresponding to a portion of the tissue b11 specified in theselection region 81. The user can freely move the display position ofthe cut display region 83 from the neighborhood of the selection region81. In addition, the user can freely move the selection region 81 of thebiopsy region image 80. As the selection region 81 is moved, the cutdisplay region 83 is also moved. As a result, the portion of the tissueb12 displayed in the cut display region 83 is changed. This displaymethod allows the user to place portions he or she wants to compare athis or her desired positions.

“Leaf-Though Display”

When the user specifies a selection region 92 having any size on abiopsy region image 91 that shows the tissue b11 and performs apredetermined operation (such as a mouse dragging or the like) for theselection region 92 as shown in FIG. 14A, the portion of the tissue b11displayed in the selection region 81 is gradually changed to the portionof the tissue b12 as shown in FIG. 14B. This display method allows theuser to compare the corresponding portions of the tissue b11 and thetissue b12 without necessity of moving his or her viewpoint. Inaddition, the ratio of regions to be compared can be adjustedcorresponding to a user's operational amount (such as mouse's draggingamount).

“Multiple Staining Color Combining Display”

A plurality of immunohistochemistries (IHCs) are performed for aplurality of corresponding biopsies as shown in FIG. 15A. Stainedportions (FIG. 15B) are superposed on a biopsy region image stained byH&E (FIG. 15C). As a result, a plurality of IHC and H&E results may besimultaneously displayed as shown in FIG. 15D. Thus, the user canobserve the IHC and H&E results at the same time and perform diagnosis.Alternatively, a plurality of IHC stained portions (FIG. 15B) thatdiffer in stained colors may be combined and displayed withoutsuperposing the stained portions on the H&E stained biopsy region image(FIG. 15C).

“Relevant Comment Display”

The user can add a diagnostic comment 111 about a predetermined portionof a biopsy region image that shows one biopsy b11 of the abovedescribed “instantaneous switch display” on the screen. When a biopsyregion image 112 that shows the other biopsy b12 is displayed, thediagnostic comment 111 made corresponding to the biopsy b11 is added tothe corresponding portion of the biopsy b12. Thus, the user can checkthe portion of the biopsy b12 corresponding to the portion of the biopsyb11 he or she has diagnosed.

As described above, the pathological image display control device 10according to an embodiment of the present disclosure can synchronouslydisplay a plurality of images.

Specifically, since corresponding portions of a plurality of images canbe simultaneously displayed, the user can improve his or her diagnosingaccuracy. Compared with the case that one biopsy is stained by aplurality of different staining methods and their colors are separated,a multiply stained image can be synchronously displayed in high quality.In addition, since a process that corrects the positions and sizes of aplurality of biopsy region images is performed, corresponding images canbe displayed without positional deviations.

In addition, staining methods that are normally performed at the sametime can be used in combinations so as to synchronously display images.In addition, a combination of images captured by a dark fieldobservation, a bright field observation, and a phase-contrastobservation cannot be simultaneously observed by ordinary microscopescan be synchronously displayed.

As the foregoing effect, the user (diagnostician) can improve his or herdiagnosing accuracy and shorten the diagnosing time.

The series of processes described above can be executed by hardware butcan also be executed by software. When the series of processes isexecuted by software, a program that constructs such software isinstalled into a computer. Here, the expression “computer” includes acomputer in which dedicated hardware is incorporated and ageneral-purpose personal computer or the like that is capable ofexecuting various functions when various programs are installed.

FIG. 12 is a block diagram showing an example configuration of thehardware of a computer that executes the series of processes describedearlier according to a program.

In the computer 200, a central processing unit (CPU) 201, a read onlymemory (ROM) 202 and a random access memory (RAM) 203 are mutuallyconnected by a bus 204.

An input/output interface 205 is also connected to the bus 204. An inputunit 206, an output unit 207, a storing unit 208, a communication unit209, and a drive 210 are connected to the input/output interface 205.

The input unit 206 is configured from a keyboard, a mouse, a microphone,or the like. The output unit 207 configured from a display, a speaker orthe like. The storing unit 208 is configured from a hard disk, anon-volatile memory or the like. The communication unit 209 isconfigured from a network interface or the like. The drive 210 drives aremovable medium 211 such as a magnetic disk, an optical disk, amagneto-optical disk, a semiconductor memory or the like.

In the computer configured as described above, the CPU 201 loads aprogram that is stored, for example, in the recording unit 208 onto theRAM 203 via the input/output interface 205 and the bus 204, and executesthe program. Thus, the above-described series of processing isperformed.

A program that the computer (CPU 201) executes can be provided as aso-called web application that allows the computer to access for examplea predetermined server on the Internet to obtain the program.

By inserting the removable medium 211 into the drive 210, the programcan be installed in the recording unit 208 via the input/outputinterface 205. Further, the program can be received by the communicationunit 209 via a wired or wireless transmission media and installed in thestoring unit 208. Moreover, the program can be installed in advance inthe ROM 202 or the recording unit 208.

It should be noted that the program executed by a computer may be aprogram that is processed in time series according to the sequencedescribed in this specification or a program that is processed inparallel or at necessary timing such as upon calling.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   10 pathological image display control device-   11 operation input unit-   12 pathological image input unit-   13 biopsy region image obtainment unit-   14 display control unit-   20 biopsy region image generation server-   21 cellular tissue region detection unit-   22 detection dictionary-   23 biopsy region grouping unit-   24 biopsy region cut unit-   25 position correction unit-   30 display-   200 computer-   201 CPU

The invention claimed is:
 1. An information processing apparatuscomprising: a processor; and a memory device, the memory device storinginstructions that cause the processor to: obtain a first pathologicalimage and a second pathological image, identify a first tissue area fromthe first pathological image based on an identification function using adetection dictionary which is pre-generated through statistical learningusing learning cellular tissue region images, identify a second tissuearea from the second pathological image based on the identificationfunction using the detection dictionary, associate the first tissue areaand the second tissue area with respective tissue samples using anautomatic clustering technique, wherein the first pathological image isat least one of a dark field image, a bright field image or a phasedifference image, wherein the second pathological image is at least oneof the dark field image, the bright field image or the phase differenceimage, wherein the first pathological image and the second pathologicalimage are images including cell tissue obtained from the same specimen,adjust at least one of shapes, orientations, positions, and sizes of thesecond tissue area, cause a display device to display the first tissuearea, and cause the display device to display a second part of thesecond tissue area on a first part of the first tissue area, wherein thesecond part of the second tissue area is corresponding to the first partof the first tissue area.
 2. The information processing apparatusaccording to claim 1, wherein the first pathological image and thesecond pathological image are images obtained by imaging tissue obtainedby staining cell tissues cut out from the same specimen with differentreagents.
 3. The information processing apparatus according to claim 1,wherein the memory device stores instructions that cause the processorto classify the first tissue area and the second tissue area into groupsbased on an image pattern.
 4. The information processing apparatusaccording to claim 3, wherein the instructions that cause the processorto adjust at least one of shapes, orientations, positions, and sizes ofthe second tissue area are instructions that cause the processor toadjust at least one of shapes, orientations, positions, and sizes of thesecond tissue area when the first tissue area and the second tissue areaare classified into the same group.
 5. The information processingapparatus according to claim 1, wherein the identification function isbased on training data comprising an image whose center is a cellulartissue region and an image whose center is a background portion.
 6. Aninformation processing method performed by an information processingapparatus, comprising: obtaining a first pathological image and a secondpathological image, identifying a first tissue area from the firstpathological image based on an identification function using a detectiondictionary which is pre-generated through statistical learning usinglearning cellular tissue region images, identifying a second tissue areafrom the second pathological image based on the identification functionusing the detection dictionary, associating the first tissue area andthe second tissue area with respective tissue samples using an automaticclustering technique, wherein the first pathological image is at leastone of a dark field image, a bright field image or a phase differenceimage, wherein the second pathological image is at least one of the darkfield image, the bright field image or the phase difference image,wherein the first pathological image and the second pathological imageare images including cell tissue obtained from the same specimen,adjusting at least one of shapes, orientations, positions, and sizes ofthe second tissue area, causing a display device to display the firsttissue area, and causing the display device to display a second part ofthe second tissue area on a first part of the first tissue area, whereinthe second part of the second tissue area is corresponding to the firstpart of the first tissue area.
 7. The information processing methodaccording to claim 6, wherein the first pathological image and thesecond pathological image are images obtained by imaging tissue obtainedby staining cell tissues cut out from the same specimen with differentreagents.
 8. The information processing method according to claim 6,further comprising classifying the first tissue area and the secondtissue area into groups based on an image pattern.
 9. The informationprocessing method according to claim 8, wherein adjusting at least oneof shapes, orientations, positions, and sizes of the second tissue areacomprises adjusting at least one of shapes, orientations, positions, andsizes of the second tissue area when the first tissue area and thesecond tissue area are classified into the same group.
 10. Theinformation processing method according to claim 6, wherein theidentification function is based on training data comprising an imagewhose center is a cellular tissue region and an image whose center is abackground portion.
 11. A non-transitory computer-readable medium havingstored thereon a computer-readable program for causing a computer to:obtain a first pathological image and a second pathological image,identify a first tissue area from the first pathological image based onan identification function using a detection dictionary which ispre-generated through statistical learning using learning cellulartissue region images, identify a second tissue area from the secondpathological image based on the identification function using thedetection dictionary, associate the first tissue area and the secondtissue area with respective tissue samples using an automatic clusteringtechnique, wherein the first pathological image is at least one of adark field image, a bright field image or a phase difference image,wherein the second pathological image is at least one of the dark fieldimage, the bright field image or the phase difference image, wherein thefirst pathological image and the second pathological image are imagesincluding cell tissue obtained from the same specimen, adjust at leastone of shapes, orientations, positions, and sizes of the second tissuearea, cause a display device to display the first tissue area, and causethe display device to display a second part of the second tissue area ona first part of the first tissue area, wherein the second part of thesecond tissue area is corresponding to the first part of the firsttissue area.
 12. The computer-readable medium according to claim 11,wherein the first pathological image and the second pathological imageare images obtained by imaging tissue obtained by staining cell tissuescut out from the same specimen with different reagents.
 13. Thecomputer-readable medium according to claim 11, wherein thecomputer-readable program further causes the computer to classify thefirst tissue area and the second tissue area into groups based on animage pattern.
 14. The computer-readable medium according to claim 13,wherein causing the computer to adjust at least one of shapes,orientations, positions, and sizes of the second tissue area, comprisescausing the computer to adjust at least one of shapes, orientations,positions, and sizes of the second tissue area when the first tissuearea and the second tissue area are classified into the same group. 15.The computer-readable medium according to claim 11, wherein theidentification function is based on training data comprising an imagewhose center is a cellular tissue region and an image whose center is abackground portion.
 16. An information processing apparatus comprising:a processor; and a memory device, the memory device storing instructionsthat cause the processor to: obtain a first pathological image andinformation regarding a first area of the first pathological image,determine a second area of a second pathological image corresponding tothe first area of the first pathological image, and cause a displaydevice to display a portion of the second area of the secondpathological image and a portion of the first area of the firstpathological image, wherein of the portion of the second area of thesecond pathological image that is displayed and the portion of the firstarea of the first pathological image that is displayed are changeable bya user operation during display, such that a ratio of a displayed amountof the second area of the second pathological image to a displayedamount of the first area of the first pathological image is changeableby the user operation during display.